Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix serialization #20

Merged
merged 3 commits into from
Mar 13, 2018
Merged

Fix serialization #20

merged 3 commits into from
Mar 13, 2018

Conversation

abitmore
Copy link
Member

Related to #15.

@pmconrad
Copy link

Could you do a full replay and measure the performance penalty of your changes? If it is insignificant I'd tend to agree with you - simplicity over performance. OTOH, if it is significant we should try to optimize.

@abitmore
Copy link
Member Author

Actually I've done a full replay, but didn't record the time used. Will do again.

@abitmore
Copy link
Member Author

abitmore commented Mar 13, 2018

@pmconrad looks like the patch doesn't affect replay at all, perhaps due to no signature verification while replaying?

  • With the whole patch:

222740ms th_a db_management.cpp:59 reindex ] reindexing blockchain
222740ms th_a db_management.cpp:65 reindex ] Replaying blocks, starting at 1...
2433130ms th_a db_management.cpp:78 reindex ] Writing database to disk at block 25162303
2441179ms th_a db_management.cpp:80 reindex ] Done
2454500ms th_a db_management.cpp:122 reindex ] Done reindexing, elapsed time: 5831.76034100000015314 sec

  • Without the patch:

1200426ms th_a db_management.cpp:59 reindex ] reindexing blockchain
1200426ms th_a db_management.cpp:65 reindex ] Replaying blocks, starting at 1...
3530797ms th_a db_management.cpp:78 reindex ] Writing database to disk at block 25165591
3536869ms th_a db_management.cpp:80 reindex ] Done
3552218ms th_a db_management.cpp:122 reindex ] Done reindexing, elapsed time: 5951.79204399999980524 sec

In above data, the latter spent even more time, but perhaps it is caused by processing the 2288 more blocks. By the way, I did enable grouped_orders plugin while replaying, so time of both is a bit longer than usual.

//Update:
No much difference after disabled grouped_orders plugin:

1852293ms th_a db_management.cpp:59 reindex ] reindexing blockchain
1852293ms th_a db_management.cpp:65 reindex ] Replaying blocks, starting at 1...
235934ms th_a db_management.cpp:78 reindex ] Writing database to disk at block 25180414
241954ms th_a db_management.cpp:80 reindex ] Done
255375ms th_a db_management.cpp:122 reindex ] Done reindexing, elapsed time: 5603.08208300000023883 sec

@pmconrad
Copy link

Thanks. That really is insignificant. Interesting.
Signature verification shouldn't matter to the question at hand. Blocks are read from disk, deserialized and processed. I'd assumed that the deserialization would be slower than before, but apparently I was wrong.

pmconrad
pmconrad previously approved these changes Mar 13, 2018
Copy link

@pmconrad pmconrad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, let's merge this.
If the json fix has conflicts I'll rebase, then we should merge it too.
Then I'll rebase my variant fixes and concentrate on completing those.

@abitmore
Copy link
Member Author

@pmconrad I got some time, so are making changes to improve performance. Will push Soon. Please check again later.

@@ -54,15 +57,15 @@ namespace fc {
inline void pack( Stream& s, const fc::log_message& msg, uint32_t _max_depth )
{
FC_ASSERT( _max_depth > 0 );
fc::raw::pack( s, variant(msg), _max_depth - 1 );
fc::raw::pack( s, variant(msg), _max_depth - 1 ); // TODO check variant depth?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FTR - Both TODOs are handled in my variant_fix branch as well, will have to resolve merge conflicts later. Leave them in for now.

@abitmore
Copy link
Member Author

Result of replay after the "optimization" commit (with frustration):

1064463ms th_a db_management.cpp:59 reindex ] reindexing blockchain
1064463ms th_a db_management.cpp:65 reindex ] Replaying blocks, starting at 1...
3429348ms th_a db_management.cpp:78 reindex ] Writing database to disk at block 25186907
3437237ms th_a db_management.cpp:80 reindex ] Done
3451642ms th_a db_management.cpp:122 reindex ] Done reindexing, elapsed time: 5987.17886300000009214 sec

Copy link

@pmconrad pmconrad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

@pmconrad
Copy link

Lol

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants